13  Time Series Classification

CautionStill under construction

This section is still under construction and will be completed in the near future. Please do not go beyond this point for now.

Time series classification belongs to the class of supervised learning and is defined as the task of assigning a label to a time series.

Again, this is a very active research area and many different methods have been proposed. For a good overview, check e.g. Faouzi (2024), which divide time series classification methods into the following categories:

• Metric: Distance-based methods, e.g. nearest neighbor classifiers with different distance measures.
• Kernels: Methods that use kernel functions to measure similarity between time series.
• Shapelets: Methods that learn discriminative subsequences (shapelets).
• Tree-based: Methods that use decision trees or random forests.
• Bag-of-Words: Methods that represent time series as a “bag of words”.
• Image: Methods that treat time series as images.
• Deep Learning: Methods that use deep learning architectures.
• Random Convolutions: Methods that use random convolutional filters.
• Ensemble: Methods that combine multiple classifiers for improved performance.

We can see that many of these categories already appeared in Section 12.1.

A simple metric-based (distance-based) baseline method for time series classification is to use a nearest neighbor classifier with a suitable distance measure for time series, e.g. dynamic time warping (DTW). This is implemented in sktime as KNeighborsTimeSeriesClassifier or KNeighborsTimeSeriesClassifierTslearn (latter is just a wrapper for the implementation in tslearn). These classifiers work similar to the KNeighborsClassifier for tabular data from sklearn, but use a distance measure suitable for time series. We have learned about euclidean distance and DTW in Section 12.2.

In this section we will have a brief look at time series forests (tree-based) in the context of time series classification.

13.1 Time Series Forests

A time series forest is an ensemble of time series decision trees. Each decision tree is built using a random set of intervals from the time series.
For each interval, summary statistics (mean, standard deviation, slope) are computed and used as features for the decision tree.
The final prediction is made by aggregating the predictions of all trees in ensemble (e.g. by majority vote for classification).

Here is a simple example showing the feature extraction of three time series:

Time Series Decision Tree Interval Example

Interval	Curve	Mean	Std	Slope
Interval 1	Curve 1	1.07	0.13	0.61
	Curve 2	-0.14	0.30	-1.17
	Curve 3	-0.95	0.05	-0.22
Interval 2	Curve 1	0.31	0.25	-1.72
	Curve 2	-0.41	0.21	1.02
	Curve 3	-0.91	0.04	0.25
Interval 3	Curve 1	-0.89	0.30	1.11
	Curve 2	-0.35	0.19	-0.04
	Curve 3	0.43	0.18	0.65
Interval 4	Curve 1	0.48	0.03	0.15
	Curve 2	0.20	0.21	0.28
	Curve 3	0.99	0.03	-0.01

TimeSeriesForestClassifier as implemented in sktime in particular uses 200 trees (n_estimators) by default and samples sqrt(m) intervals per tree, where m is the length of the time series.
More configurable tree based ensembles are provided with ComposableTimeSeriesForestClassifier.

13.1.1 Example

We will use the same dataset as in the clustering example from ?sec-act3-cluster-examples, the Trace dataset.

import matplotlib.pyplot as plt
import plotly.graph_objs as go
import plotly.io as pio
import seaborn as sns

from sktime.classification.ensemble import ComposableTimeSeriesForestClassifier
from sktime.classification.interval_based import TimeSeriesForestClassifier
from sklearn.metrics import confusion_matrix
from tslearn.datasets import CachedDatasets

pio.renderers.default = "notebook"  # set the default plotly renderer to "notebook" (necessary for quarto to render the plots)

X_train, y_train, X_test, y_test = CachedDatasets().load_dataset("Trace")

# Fix shape for TimeSeriesForestClassifier
X_train = X_train[:, :, 0]  
X_test = X_test[:, :, 0]

Note: Here, the dataset already comes split into training and test set. In practice, you would want to do a proper train-test split on your own dataset.

We set up the classifier and train on the training set \((X_{train}, y_{train})\) as follows.

clf = TimeSeriesForestClassifier(random_state=42)  # here, we set the random state for reproducibility
clf.fit(X_train, y_train)

TimeSeriesForestClassifier(random_state=42)

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

Having a fitted classifier model, we can predict the labels of the previously unseen test set \(X_{test}\).

y_pred = clf.predict(X_test)

The confusion matrix shows that the classifier performs extraordinarily well on this dataset, achieving an accuracy of almost 100% on the test set.

cm = confusion_matrix(y_test, y_pred)
plt.figure(figsize=(6, 5))
sns.heatmap(cm, annot=True, fmt='d', cmap='Blues')
plt.xlabel('Predicted')
plt.ylabel('True')
plt.title('Confusion Matrix')
plt.show()

The following shows the classified time series from the test set, colored by their predicted label.

base_colors = {1: '#0072B2', 2: '#E69F00', 3: '#009E73', 4: '#D55E00'}

fig = go.Figure()

labels = sorted(set(y_test))

for label in labels:
    idx = (y_test == label)
    series = X_test[idx]
    n = series.shape[0]
    base_color = base_colors[label]
    legendgroup = f"Label {label}"
    # Plot each time series with low opacity
    for i in range(n):
        fig.add_trace(go.Scatter(
            y=series[i],
            mode='lines',
            line=dict(color=base_color),
            opacity=0.2,
            showlegend=False,
            legendgroup=legendgroup
        ))
    # Plot the mean line for each label
    fig.add_trace(go.Scatter(
        y=series.mean(axis=0),
        mode='lines',
        line=dict(color=base_color, width=3),
        name=legendgroup,
        opacity=1,
        legendgroup=legendgroup
    ))

fig.update_layout(
    title="Test Set Time Series by Label",
    xaxis_title="Time",
    yaxis_title="Value",
    legend_title="Label",
    width=900,
    height=600
)
fig.show()

Let us also visualize the two misclassified time series:

misclassified_idx = y_pred != y_test
misclassified_series = X_test[misclassified_idx]
misclassified_true = y_test[misclassified_idx]
misclassified_pred = y_pred[misclassified_idx]

fig = go.Figure()

for i in range(len(misclassified_series)):
    true_label = misclassified_true[i]
    pred_label = misclassified_pred[i]
    fig.add_trace(go.Scatter(
        y=misclassified_series[i],
        mode='lines',
        line=dict(color=base_colors[true_label]),
        name=f'True: {true_label}, Pred: {pred_label}',
        opacity=0.7
    ))

fig.update_layout(
    title="Misclassified Test Set Time Series",
    xaxis_title="Time",
    yaxis_title="Value",
    legend_title="True/Predicted Label",
    width=900,
    height=600
)
fig.show()

The two misclassified time series actually belong to class 2 but were predicted class 1.

Exercise 13.1 (Discussing results)  

Why might these two time series have been misclassified?

Faouzi, Johann. 2024. “Time Series Classification: A review of Algorithms and Implementations.” In Time Series Analysis - Recent Advances, New Perspectives and Applications, edited by Jorge Rocha, Sandra Oliveira, and Cláudia Viana, 298. https://doi.org/10.5772/intechopen.1004810.